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Introduction 




T his report is part of a series of studies related to college going by 
Chicago Public Schools (CPS) students. Preparing more students for 
college has been a hallmark of the current CPS administration, as evidenced 
in the creation of the Department of Postsecondary Education and Student 
Development, its implementation of many programs designed to help more 
students reach and succeed in college, and the system-wide goal of increasing 
the number of students with scores close to the national and state averages. 
This study looks closely into the CPS goal of having more students score 20 
or better on the ACT. Part I examines the relationship between prior test 
scores including PLAN, EXPLORE, and the eighth grade ISAT — and ACT 
scores — and finds that scores on the earlier tests are highly predictive of ACT 
scores (see page 6 for explanation of tests) . To the surprise of many, though, 
we also find that most students who meet the state standards on the eighth 
grade ISAT have little chance of reaching 20 on the ACT. Only students who 
reach the Exceeds Standards category have a good chance of scoring a 20 or 
better. This suggests major misalignment between our expectation for what 
students should know and be able to do at the end of elementary school and 
whether or not they are on track for college readiness. Part II of this study 
reviews some previous Consortium on Chicago School Research (CCSR) 
research on what matters for receiving good scores on the ACT and also in- 
troduces some new findings on this topic. Both the prior and current research 
clearly identify two major factors that influence ACT scores. Regardless of 
their background or previous test scores, students score higher on the ACT 
when they attend high schools with strong academic cultures and when they 
have high academic performance as evidenced by good grades. Students have 
better ACT scores when they attend good high schools, as we describe further 
in this report, and when they successfully engage in coursework. 
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ACT Scores in Chicago: 
Reaching for a Goal of 20 



A s more and more high school students across the country aspire to attend 
J. X. and graduate from college, educators are looking more closely at the 
factors that facilitate or hinder college going. Researchers at the Consortium 
on Chicago School Research (CCSR) have very carefully examined the process 
of preparing for postsecondary education in Chicago Public Schools (CPS) 
and have released three significant studies on the topic. The first, From High 
School to the Future: A First Look at Chicago Public School Graduates’ College 
Fnrollment, College Preparation, and Graduation from Four-year Colleges, 
found that far too many CPS high school students were simply not prepared 
to attend college because of chronically low grades and test scores.^ These 
students had neither the academic preparation for college nor the “habits of 
mind” needed for independent and challenging college courses. 

The second study. From High School to the Future: Potholes on the Road 
to College, showed that even well prepared high school students stumbled 
on the roadway to college because of lack of information and lack of 
access to knowledgeable and helpful people to guide them.^ Many of those 
who did make it to college often enrolled in weak institutions in spite of 
having the qualifications for attending better colleges. The most recent 
study. From High School to the Future: ACF Preparation — Foo Much, Foo 
Late, described the massive amount of time and energy that high school 
teachers and students spend in preparing for the ACT in the spring of 
junior year.^ Unfortunately, the test prep activities detract from high quality 
classroom instruction and actually have a negative impact on students’ ACT 
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scores. This study found that academic preparation, as 
measured hy grades in subject matter classes (English, 
math, science, and social science), has the strongest 
positive effect on ACT scores. 

There are clearly two groups of students with distinct 
problems: one with insufficient academic preparation 
for college work, and another with the right academic 
skills but that lacks sufficient human and social capi- 
tal to attend good colleges. CPS is working on both 
of these. To address the human and social capital 
problem, the CPS Department of Postsecondary 
Education and Student Development has instituted 
numerous programs to provide information and sup- 
port to high school students so that they will learn more 
about and apply to a range of colleges. Regarding the 
first problem, the school district has initiated a wide 
range of efforts to improve student achievement in 
high school. Describing these programs is beyond the 
scope of this report, but it is clear that low test scores 
are a major concern for CPS and numerous efforts are 
underway to improve instruction and raise high school 
test scores. 

To provide a clear goal across the many different 
programs working on improving student achievement, 
CPS is aiming to increase the number of students in 
each high school who achieve a composite score of at 
least 20 on the ACT. As a signal of the importance of 
this indicator, the most recent high school score cards 
include the percentage of students who scored 20 or 
higher on their ACT composite.'* A composite score of 
20 on the ACT is somewhat lower than both the state 
and national average scores (20.5 for 2007 graduates 
in Illinois and 21.2 nation-wide). 5 In terms of predict- 
ing future college success, an ACT score of 20 is also 
somewhat low. According to research conducted at 
ACT, students who attain specific benchmark scores 
on the four subject area tests will have a 50 percent 
chance of earning a grade of B or better in a fresh- 
man college level course and a 75 percent chance of 
earning a C or better. The ACT benchmark scores are 
18 for English, 22 for Math, 24 for Science, and 21 
for Reading. The Reading benchmark of 21 may be 
somewhat low. Another ACT study suggests that stu- 
dents need complex reading skills for college and that 
these are demonstrated by even higher ACT scores.^ 



Although there are no benchmark scores for composite 
ACT scores, three of the four benchmarks are 21 or 
higher, and together they average out to about 21.7 

CPS chose this specific score of 20, in spite of be- 
ing a little lower than state and national averages and 
the benchmark scores, because CPS graduates with 
this score or better — and good grades — will have a 
chance of being accepted into many Illinois state uni- 
versities. Eor example, at Southern Illinois University 
at Edwardsville, Northern Illinois University, and the 
University of Illinois at Chicago a composite ACT score 
of 20 ranks at about the 25th percentile for the college 
class of 2005.® The average freshmen at these colleges 
had somewhat higher ACT scores, but still a 20 would 
give a CPS graduate some moderate chance of gaining 
admission to these institutions. CPS students with a 
20 on the ACT would also have access to many of the 
historically black colleges and universities, such as Eisk 
University, where the average ACT score for entering 
students was 20. 

The aim of this report is to help CPS, its schools, 
and its students see what the pathway to 20 looks like 
and how the district can successfully guide many more 
students down this path. The first part of the paper 
describes the pathway in relation to students’ previous 
PEAN, EXPEORE, and ISAT scores. In the second 
part of the paper, we draw from multiple CCSR studies 
and present two consistent sets of research findings. 
Eirst, students do better in high schools where there is 
a strong academic culture. Second, students do well on 
the ACT when they earn good grades by responding 
positively to good instruction. 

ACT Scores in CPS 

ACT scores in the Chicago Public Schools are quite 
low by any standard. When we combine all ACT 
scores from all schools in CPS from the spring 2005, 
2006, and 2007 eleventh grade administrations of 
the Prairie State Achievement Examination, for a 
total of 55,221 scores, we see a skewed distribution, 
where most of the students receive scores on the lower 
end of the scale and few students receive the higher 
scores. The most typical score for CPS juniors in the 
spring of 2005, 2006, and 2007 was 15. See Eigure 1. 
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More than half of students (52.3 percent) scored 16 
or helow; fewer than half (47.7 percent) scored 17 or 
higher. The average score was 17.2. Only one in four 
students — 25 percent — in the combined three cohorts 
of students who were juniors in 2005, 2006, and 2007 
scored 20 or better, the CPS goal.^ More than one-half 
(56 percent) of these relatively high achieving students 
were concentrated within five CPS high schools. 

If the goal were set higher, say to 21, closer to the 
national average (which counts mostly college-bound 
students), we would see only one in five or 21 percent 
of CPS juniors reaching that goal. If it were higher 
yet, say 23 (the average incoming ACT score of college 
freshmen at University of Illinois at Chicago, Spelman 
College, and Howard University, for example) we 
would find only 10 percent of CPS juniors scoring at 

FIGURE 1 

About one-quarter of CPS juniors score 20 or above on the ACT 




Composite ACT Score 

■ Students with ACT <20 ■ Students with ACT > = 20 

Note: This sample includes 55,221 students who took the ACT as eleventh graders In 2005, 2006, and 2007. 



this level. Though we are stressing how low the most 
typical scores are in CPS, it is only fair to point out that 
there are many high scoring students as well. Though 
it is hard to see on this figure, 805 students had scores 
of 30 and better, with 10 CPS juniors achieving a 
perfect score of 36, and another 40 one point shy of 
perfect with a 35. In addition, the CPS average ACT 
scores have improved steadily for the past several years, 
with a slowdown in the most recent years. According 
to documents posted on the CPS website, the aver- 
age composite ACT score has increased from 16.1 in 
2001 to 17.2 in 2007.^' The recent slowdown in score 
increases is apparent in the sample we are looking at 
here, where the average score was 17.1 in 2005, 17.1 in 
2006, and 17.2 in 2007. 
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Students’ Scores on EXPLORE and PLAN 

To understand the pathway to 20 — and its detours 
and exits — it is helpful to look hack from the ACT to 
see students’ progress on prior tests. In addition to the 
ACT, all CPS high school students also take two other 
tests developed hy ACT. These tests, the EXPLORE 
and PLAN, along with the ACT, make up EPAS — 
the Education Planning and Assessment System. 
They are designed to measure student academic de- 
velopment over time and help students make decisions 
about important transitions in their lives — from ele- 
mentary to high school and heyond into postsecondary 
education and the workplace. The system is intended 
to provide students and their families, teachers, and 
counselors, and school personnel with useful informa- 
tion to guide their decision making and planning. In 
CPS, all ninth graders take the EXPLORE in the fall, 
all tenth graders take PLAN in the fall, and all eleventh 
graders take the ACT in the spring. 1 3 

These tests are all scored on approximately the 
same scale to make it easy — and appropriate — to 
measure improvement from one test to the next.'“^ 
The tests get more difficult (and longer) , moving from 
EXPLORE, to PLAN, to ACT; and the maximum 
possible scores increase correspondingly from 25 on 
EXPLORE, to 32 on PLAN, to 36 on ACT. ACT is 
considerably more difficult than both EXPLORE and 
PLAN, requiring more complex problem solving and 
critical thinking skills. From High School to the Future: 
ACF Preparation — Foo Much, Foo Late shows several 
striking examples of how the ACT makes much greater 
cognitive demands of students than EXPLORE and 
PLAN. In general, the ACT requires greater analytic 
skills; the ability to infer meaning from unfamiliar 
words and situations; and more multi-step, as opposed 
to single-step, problem solving. The study suggests 
that large numbers of CPS students are simply un- 
prepared for and surprised by these demanding tasks. 
They have not had sufficient experiences, either in their 
regular classroom instruction or in test preparation ac- 
tivities, that require these greater cognitive demands. 

In this study, we track our three cohorts of juniors 
back to their prior EXPLORE scores in ninth grade 
and PLAN scores in tenth grade. (Because CPS has 



discontinued the eleventh grade PLAN, we are not 
including it here, though we do refer to eleventh grade 
PLAN later.) When we match student scores like this, 
we invariably “lose” students from our sample because 
many of them did not take all of the previous tests for 
any number of possible reasons. They may not have 
been enrolled in CPS, or they may have missed taking 
the test. Some students fall out of our matched sample 
because they are not promoted each year. Our original 
sample of 55,221 is reduced to 42,456 students — a 
drop off of about 23 percent. These students with the 
matched scores are a somewhat more select group than 
the students who do not have matched scores. This 
sample only includes students who were in eleventh 
grade in 2005, 2006, or 2007; in tenth grade the pre- 
ceding year; and in ninth grade the year before (2003, 
2004, or 2005). In other words, these are students 
who are “on-track” through eleventh grade. The mean 
ACT score for this matched sample is 17.6, compared 
to the slightly lower average of 17.2 for the full sample 
of ACT scores. Twenty-eight percent of these students 
reached 20 on the ACT, as compared to the 25 percent 
in the full sample. 

FIGURE 2 

CPS students see much smaller improvements from PLAN to 




EXPLORE PLAN ACT 



How to Read a Box Plot: Box plots display Information about both the distribution and central 
tendency of the test scores. The median is marked by the white line in the middle of the box. The 
mean (arithmetic average) is the white dot inside the box. The shaded box includes the scores 
of the middle 50 percent of students. The top of the box marks off the score for students at the 
75th percentile, and the bottom of the box marks the score for students at the 25th percentile. 
The lines extending up and down from the box show the range for the top and bottom scoring 
students. The scores of about 99 percent of all students fall between these lines. 

Note: This sample includes 42,456 students who were in eleventh grade in 2005, 2006, and 
2007; in tenth grade the year before; and in ninth grade the year before that. 
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Figure 2 shows the distrihution of composite scores 
on the ninth grade EXPLORE, tenth grade PLAN, and 
eleventh grade ACT for these three combined cohorts. 
We use hox plots to show these scores because they 
display information about both the distribution and 
central tendency of the scores. The median is marked 
by the line in the middle of the box. The mean (arith- 
metic average) is the circle. The upper bar on the box 
marks off the score for students in the 75th percentile, 
and the lower bar on the box marks the score for stu- 
dents in the 25th percentile. 

The average EXPLORE score is 15. As the box plot 
shows, a full one-quarter of students score 13 or below, 
and only the top quarter of students score a 17 or better. 
(We shall see later that an EXPLORE score of 17 is 
an important benchmark for the goal of achieving a 
20 or better on the ACT.) The average PLAN score for 
the same students one year later is 16.2; and, as noted, 
the average ACT score is 17.6. Again, even in this 
more select group of students who took all three tests, 
only somewhat more than one-quarter are reaching 
the goal of 20 on the ACT; and a full one-quarter of 
students are scoring 14 and below. 

Because these scores are matched and on approxi- 
mately the same scale, we can calculate the average 
amount of improvement that students make from one 
test to the next. On average, CPS students increased 
their scores by 1.2 points from EXPLORE to PLAN, 
and 1.4 points from tenth grade PLAN to eleventh 
grade ACT. 

To help school officials and students make use of 
and understand the gains across EPAS, ACT publishes 
tables that show the scores of a large national sample of 
students who took EXPLORE in the fall of their ninth 
grade and then also took PLAN in the fall of tenth 
grade. The table shows how many students from na- 
tional administrations of EXPLORE obtained specific 
PLAN scores one year later, CPS used these tables to 
create “expected gains” from EXPLORE to PLAN. The 
expected gain is actually the median gain achieved by 
all students in the national sample on PLAN for every 
possible EXPLORE score. For example, about 18,000 
students in this national sample obtained a composite 
score of 14 on EXPLORE in the ninth grade. One year 



later, the median PLAN score for these students was 
16; so 2 points is considered the “expected gain” for an 
EXPLORE score of 14. In the same fashion, there are 
expected gains for each possible PLAN score. There 
is a similar table for students who took PLAN in the 
fall of tenth grade and then took ACT in the spring of 
eleventh grade. Again, CPS calculated expected gains 
for each possible PLAN score using the same technique. 
The expected gain is the difference between a given 
possible PLAN score and the median of the obtained 
ACT scores. 

In general, there is a “U-shaped” relationship be- 
tween prior score and expected gain. Very low scoring 
students are likely to make relatively high gains, average 
students make smaller gains, and the higher scoring 
students also make greater gains (though not as great 
as the lowest scoring students). There are expected 
gains in each of the four subject areas, as well as in 
the composite, although these are not obtained from 
published ACT sources. We can compare the observed 
gains on the composite score among CPS students to 
the expected gains from the ACT national sample. 

As shown in Figure 2, the average composite score 
gain from EXPLORE to PLAN for the CPS ninth 
grade students is 1.2. This is slightly lower than the av- 
erage expected gain for these same students of 1.3. We 
calculated this by averaging the expected gain for each 
of the 42,456 students in this matched sample. 

The average gain from PLAN to ACT for the tenth 
grade students shown here is 1.4. This compares un- 
favorably to the average expected gain of 1.9 for these 
same students. (Again, we calculated this average 
expected gain by averaging the expected gain for all 
of the obtained PLAN scores in the CPS sample.) 
The actual CPS system-wide gain from tenth grade 
PLAN to eleventh grade ACT is lower than the average 
expected gain by 0.5 points. 

CPS students look only slightly worse than students 
nationally on EXPLORE to PLAN improvements, but 
they look markedly worse going from PLAN to ACT. 
They are not gaining in academic skills at nearly the 
same rate as students of comparable achievement levels 
across the country. Chapter 2 of this paper explores 
the reasons why. 
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Improvements from EXPLORE to PLAN 
to ACT 

Because we want to understand more about the experi- 
ences of students who receive a composite score of 20 
or better on the ACT, it is useful to examine the prior 
histories of those who achieved this goal. Figure 3 is 
much like the previous one, except that it breaks out 
two groups of students — those who made it to 20 and 
those who did not — and displays their prior EXPLORE 
and PLAN scores. Notice that there is no overlap of 
the box plots of ACT scores for the two groups of 
students, since these are mutually exclusive groups. 
(See Table 1 for more detail.) The top line represents 
EXPLORE, PLAN, and ACT scores for the 28 percent 
of the students in the matched sample who reached 20 
or more. The bottom line shows the same test scores 
for the remaining 72 percent of students who did not 

TABLE 1 

Distribution of ERAS scores and test gains from freshman to 
junior year 

This table shows the how the middle 50 peroent of students 
scored on the EXPLORE, PLAN, and ACT tests, as displayed 
in Figures 2 and 3. It reflects scores at the lower, middle, and 
higher ranges for three groups: all students, students who 
scored at least a 20 on the ACT composite, and students who 
scored below 20. This data includes three cohorts of students 
who took the ACT as juniors in 2005, 2006, and 2007. 



All students 


EXPLORE 


PLAN 


ACT 


75th percentile 


17 


18 


20 


50th percentile 


15 


16 


17 


25th percentile 


13 


14 


14 


Mean 


15 


16.2 


17.6 


Students who reached 20 


EXPLORE 


PLAN 


ACT 


75th percentile 


20 


22 


25 


50th percentile 


19 


20 


22 


25th percentile 


17 


18 


21 


Mean 


18.8 


20.1 


23.4 


Students who did not reach 20 


EXPLORE 


PLAN 


ACT 


75th percentile 


15 


16 


17 


50th percentile 


14 


15 


15 


25th percentile 


12 


13 


14 


Mean 


13.6 


14.7 


15.3 



reach the desired outcome. There are several striking 
aspects to this graph. Eirst and most obvious, these two 
groups of students are very different from each other. 
This is not at all surprising given that the groups were 
defined on the basis of their junior year ACT scores, 
but they also look remarkably different on their previ- 
ous scores. The difference in average ACT scores for 
the two groups is very large — 8.1 points on the ACT 
scale, with the stronger group of students having an 
average ACT composite score of 23.4 compared to the 
weaker group average of 15.3. This difference extends 
back to both the PLAN and EXPLORE as well. On 
PLAN, the average score for the higher scoring group is 
20.1, compared to 14.7 for the lower scoring students. 
On EXPLORE, the top group scored 18.8 and the 
bottom group scored 13.6. On all three tests, these are 
substantial and very large differences. 

Of course we expect there to be a big difference on 
the ACT scores since we created the two groups to be 
different, but the differences on PLAN and EXPLORE 
are also surprisingly great. The graph also points to 
other important and major differences between these 
two groups of students. It appears that both groups 
of students made similar gains from EXPLORE to 
PLAN — 1.3 points for the stronger group and 1.1 for 
the weaker group. These differences are more marked 
when we compare them to average expected gains, 
however. The students who reached 20 on the ACT 
(represented as the top line in Eigure 3) made greater 
than their average expected improvements (1.3 vs. 1.1). 
In contrast, the students who did not reach 20 on the 
ACT made less than their average expected gain (1.1 
vs. 1.4). 

The differences are even greater from PLAN to 
ACT. The students who achieved the ACT score of 
20 made very strong gains from PLAN to ACT — on 
average they gained 3.3 points, higher than their av- 
erage expected gain of 2.7. The less successful group 
of students gained only 0.6 points from PLAN to 
ACT, much less than their average expected gain of 
1.6 points. These students are basically “treading 
water” in terms of how much they are learning in 
their sophomore and junior years of high school, and 
they are actually falling behind in terms of expected 
gains. So we have two groups of students who start off 



8 



From High School to the Future: The Pathway to 20 



FIGURE 3 

Students who score at least 20 exceed expected gains from 
PLAN to ACT, while lower scoring students gain very little 




EXPLORE PLAN ACT 

■ Students with ACT <20 ■ Students with ACT > = 20 



How to Read a Box Plot: Box plots display information about both the distribution and 
oentral tendency of the test scores. The median is marked by the white line in the middle of the 
box. The mean {arithmetic average) is the white dot inside the box. The shaded box includes 
the scores of the middle 50 percent of students. The top of the box marks off the score for 
students at the 75th percentile, and the bottom of the box marks the score for students at 
the 25th percentile. The lines extending up and down from the box show the range for the top 
and bottom scoring students. The scores of about 99 percent of all students fall between 
these lines. 

Note: This sample includes 42,456 students who were in eleventh grade in 2005, 2006, and 
2007; in tenth grade the year before; and in ninth grade the year before that. 

differently on EXPLORE and then the differences 
between them only increase over time, with the biggest 
increase occurring between sophomore year PLAN and 
junior year ACT. The groups differed by about 5 points 
on EXPLORE, only slightly more on PLAN, and 8 
points on ACT. Clearly these two groups are having 
very different experiences in school, with one group 
learning at an accelerated pace and the other group at 
a much lower pace. The differences between the groups 
increase over time, with the greater increase occurring 
between PLAN and ACT.^^ 

Even though we are pointing out the differences be- 
tween these two groups of students, it is also important 
to note that there are similarities. There are students 
who look relatively similar to each other on EXPLORE 
and PLAN scores, but who end up with different ACT 
scores. For example, the top quarter of students who 
did not reach 20 on the ACT have similar EXPLORE 
scores to the bottom quarter of students who did reach 



20 on the ACT. Why did two groups of comparable 
students end up with such different outcomes from 
each other? We return to this key topic in the second 
part of this paper to present evidence and provide an 
explanation of why attending a strong high school and 
getting good grades there lead to higher test scores 

Predicting ACT Scores from Eighth and 
Ninth Grade Test Scores 

Students’ ninth grade EXPLORE composite scores 
say a lot about the likelihood that they will reach a 20 
or better on the ACT. Virtually no students with very 
low scores (15 and below) on EXPLORE make it to 
20 on ACT. About 30 percent of students who scored 
16 on the ninth grade EXPLORE reached 20 on the 
ACT. An EXPLORE score of 17 is the “tipping point,” 
where students have greater than a 50 percent chance 
of reaching 20 on the ACT. Virtually all students 
with high EXPLORE scores (18 and above) make it 
to 20 on the ACT. Figure 4 shows this relationship in 
a rather dramatic way. The curve is flat at either end 
of the EXPLORE distribution and very steep in the 
middle. Low scores on EXPLORE correspond with low 
chances of reaching 20, mid-range scores correspond 
with mid-range chances of reaching 20, and high 
scores on EXPLORE correspond with high chances 
of reaching 20 on ACT. 

Now we step back another year and look at the 
relationship between eighth grade ISAT scores and 
eleventh grade ACT scores.^^ All students in Chicago, 
as well as all others in the state of Illinois, take these two 
tests. Figure 5 shows the relationship between eighth 
grade ISAT Math scale scores and ACT composite 
scores in the eleventh grade. (We chose to show the 
ISAT Math scores because Math is a slightly stronger 
predictor of the ACT composite than ISAT Reading 
scores, but the relationship holds equally well with 
Reading scores.) Again we see a strong relationship 
between prior test scores and ACT performance. Note 
that this graph looks very similar to Figure 4, which 
shows the relationship between composite ninth grade 
EXPLORE and ACT scores. 

The ISAT is scored on a continuous scale ranging 
from 200 to 320 for eighth grade. Students who scored 
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FIGURE 4 



Students who score at least a 17 on EXPLORE have a good likelihood of reaching 20 on ACT 
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Note: This sample includes 42,456 students who were in eleventh grade in 2005, 2006, and 2007; in tenth grade the year before; and in ninth grade the year before that. 



low on their eighth grade ISAT have little chance of 
reaching 20 on the ACT. As ISAT scores go up, so does 
the prohahility of obtaining a score of 20 on the ACT. 
An ISAT Math score of about 265 results in about a 
20 percent chance of reaching a 20 on the ACT three 
years later. An ISAT score of 280 corresponds to about 
a 50 percent chance, and an ISAT score around 290 
corresponds to about a 75 percent chance. 

Most people are unfamiliar with the ISAT scale 
scores and are used to seeing ISAT results expressed 
in terms of “performance standards” or “achievement 
levels.” Using well known standard setting procedures, 
the Illinois State Board of Education has identified cut 
points on the ISAT continuous score scale to differenti- 
ate one performance level from another.^^ The lowest 
performance level is called “academic warning,” then 
“below standards,” then “meets standards,” and finally 
“exceeds standards.” What is called “meets standards” 
in Illinois is usually called “proficient” elsewhere. 

As shown in Figure 5, students who just meet the 
state standard (set at 246 or higher) have virtually no 



chance of getting to 20 on the ACT. A student with 
a scale score of 250 has about a 5 percent chance of 
scoring 20 on the ACT three years later. Only with 
scores of 280 and higher do students have a 50 percent 
chance or better of scoring 20 on the ACT. For those 
students who just inch their way into the Exceeds 
standards category with a score of 288, the probability 
of reaching 20 is about 62 percent. 

To further describe the relationship between eighth 
grade ISAT scores and the ACT, we calculated the 
average ACT composite score for students by the per- 
formance level on the eighth grade Math ISAT. This 
is shown in Figure 6. The average ACT score for stu- 
dents in “academic warning” is 12.5; for students “below 
standards” it is 14.4. As the box plots show, almost no 
students in either of these categories score 20 on the 
ACT. The average ACT score for students who “meet 
standards” is 17.5 (very close to the CPS average), and a 
very small portion of them reach 20. Only students who 
exceeds standards have an average ACT score above 20 
(it is 23.3), and have most of them reach 20. 
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FIGURE 5 

Only students who exceed standards on their eighth grade ISAT math tests have at least a 62 percent chance of scoring 20 on their ACT 




220 240 260 280 300 320 340 

ISAT Math Score 



Note: This sample includes 40,099 students who took the ACT as eleventh graders in 2005, 2006, and 2007 who also had eighth grade ISAT scores three years earlier. 



FIGURE 6 

Only a small portion of students who meet standards on 
eighth grade Math ISAT reach a 20 on the ACT, and one-fourth 
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How to Read a Box Plot: Box plots display information about both the distribution and central 
tendency of the test scores. The median is marked by the white line in the middle of the box. The 
mean (arithmetic average) is the white dot inside the box. The shaded box includes the scores 
of the middle 50 percent of students. The top of the box marks off the score for students at the 
75th percentile, and the bottom of the box marks the score for students at the 25th percentile. 
The lines extending up and down from the box show the range for the top and bottom scoring 
students. The scores of about 99 percent of all students fall between these lines. 
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High Correlations among Tests 

Even though the ISAT standards are misaligned 
with the ACT results, the tests are still highly 
correlated. In this sample of CPS students, the 
eighth grade ISAT Math score is correlated .81 
with the eleventh grade composite ACT. The cor- 
relation between the composite EXPLORE and 
the composite PLAN is .88, and the correlation 
between the composite PLAN and ACT is .89. 

Because of these high correlations, we see simi- 
lar patterns across tests. The eighth grade ISAT 
predicts scoring 17 on the ninth grade EXPLORE 
the same way that it predicts scoring 20 on the 
ACT. This, and the prior evidence showing that 
an EXPLORE score of 17 is the “tipping point” 
for a good chance of reaching 20 on the ACT, 
suggests that 17 can serve as a realistic interim 
benchmark on the pathway to 20. 
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Although students may indeed he meeting the 
performance standards for Illinois, there is a major 
disconnect between those performance levels in eighth 
grade and scoring well on the ACT three years later. 
The eighth grade performance standards look hack at 
the work that students should he able to do accord- 
ing to the state academic learning standards. But the 
evidence we have here suggests that the eighth grade 
standards are not well aligned with the work that stu- 
dents will need to be able to do in order to succeed in 
college. Perhaps we are sending students and schools 
the wrong message about the adequacy of elementary 
student academic preparation, especially for the vast 
majority of students who have their eyes on college in 
the future. Students, their parents, and their schools 
are being told that they meet state standards for eighth 
grade achievement; yet they have virtually no chance 
of reaching a score of 20 on the ACT, which we note 
is an admittedly low bar. As described above, this is a 



low goal but one that will give many CPS graduates a 
decent shot at a somewhat selective college or university. 
Meeting state standards, at least relative to the more 
rigorous and demanding ACT, is insufficient. It takes 
a score well into the Exceeds category on the eighth 
grade ISAT to have a relatively good shot at scoring 
well on the ACT in eleventh grade. 

The technical demands of setting performance 
standards /achievement levels can be complicated by 
the political demands. States (and schools) are under 
pressure to make adequate yearly progress as defined 
by No Child Left Behind based on performance on the 
state tests. Lower achievement levels, especially for the 
“meets standards” level (usually called “proficient”), 
make it easier to meet adequate yearly progress. Some 
observers have suggested that Illinois deliberately 
set elementary school performance standards low, or 
lowered them in 2006, so that student achievement 
looks better than it actually is.^3 
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How CPS Students Stay on 
the Pathway to 20 

T hus far we have shown how strongly prior test scores predict students’ 
ACT scores. Whether we look at eighth grade ISAT scores or ninth 
grade EXPLORE scores, we find that students who score poorly have little 
chance of reaching 20 on the ACT and students who score well have a good 
chance. Very few students who start out very low on either the ISAT or the 
EXPLORE will make it to 20 on the ACT. Students with high scores on ISAT 
and EXPLORE are likely to reach 20. Our ability to accurately predict ACT 
scores by previous achievement test scores does not mean they are determined 
by these prior scores, however. There are many students who start in the same 
place but end up different from each other. It is students’ school experiences 
that play such a strong role in determining academic achievement. 

In the remainder of this paper we argue that there are two major fac- 
tors over and above students’ prior achievement test scores that determine 
how well they do on the ACT. These are the quality of the high school, as 
measured by its academic culture; and the quality of work that students do, 
as measured by their grades. As we demonstrate below, these two factors have 
significant influence on the amount of improvement students make over time 
from one test score to another. 

The Impact of Academic Culture and Good Grades 
on Student Achievement 

This report includes examples of groups of students who look very similar to each 
other in terms of their achievement test scores at one point in time, yet they end up 
following divergent paths going forward. Looking back to Figure 3 (see page 9), 
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note the overlap in EXPLORE scores between the 
students who made it to 20 and those who did not. 
The lower quarter of students who made it to 20 look 
very much like the top quarter of students who did not 
get to 20. Why did one group make it and the other 
did not? We can also go back to Figure 4 (see page 10) 
and note the students who scored 17 on PLAN. Why 
did about half reach 20 while the other half did not? 
In both of these comparisons, we find the same two 
primary factors responsible for the higher ACT scores: 
attending high schools with strong academic cultures 
and earning high grades in courses. 

Three recent Consortium studies have presented 
compelling evidence about the positive effects of 
strong academic climates on several important student 
outcomes. In What Matters for Staying On-Track and 
Graduating in Chicago Public High Schools, the authors 



.1 



Items from Four Key Survey Measures 

1. Student-Teacher Trust measures students’ percep- 
tions about the quality of their relationships with 
teachers. Questions ask students if teachers care 
about them, keep promises, listen to their ideas, 
and try to be fair. High levels indicate that there is 
trust and open communication between students 
and teachers. 

How much do you agree with the following 
statements about your teachers: 

• My teachers really care about me. 

• My teachers always keep their promises. 

• My teachers always try to be fair. 

• I feel safe and comfortable with my teachers at 
this school. 

• When my teachers tell me not to do something, 
I know they have a good reason. 

• My teachers treat me with respect. 

Possible Answers: Strongly Disagree, Disagree, 
Agree, Strongly Agree 



show how the academic culture of high schools affect 
freshman attendance rates, freshman failure rates, 
and freshman grade point averages. Although there 
are some differences in which aspects of the positive 
academic culture in high schools impact which out- 
comes, there is also general consistency. For example, 
indicators of positive student and teacher relation- 
ships have strong positive impacts on all three student 
outcomes (attendance, failures, and grade point aver- 
age) during the freshman year. More specifically, in 
schools where there are strong reports from students on 
student-teacher trust, freshmen have 2.30 fewer days 
absence per semester, 0.78 fewer failures per semester, 
and their grade point averages are 0.23 points higher 
than students with similar background characteristics 
(including prior test scores) who attend similar schools. 
These analyses very carefully control for other possible 



2. School-Wide Academic Press for the Future measures 
students’ views of school norms of academic expecta- 
tions. Students report on the degree to which all stu- 
dents are expected to work hard to stay in school, to plan 
for their futures, and to have high personal aspirations 
for their lives after graduation. High levels indicate that 
the school expects all students to work hard, to stay in 
school, and to plan seriously for their futures. 

How much do you agree with the following about your 
high school: 

• Teachers make sure that all students are planning 
for life after graduation. 

• Teachers work hard to make sure that all students 
are learning. 

• High school is seen as preparation for the future. 

• All students are encouraged to go to college. 

• Teachers pay attention to all students, not just the 
top students. 

• Teachers work hard to make sure that students 
stay in school. 

Possible Answers: Strongly Disagree, Disagree, 
Agree, Strongly Agree 
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differences between students and schools, so we are 
pretty confident that we are isolating the positive effects 
of high levels of student-teacher trust on attendance, 
failures, and grade point averages. It is not just the case 
that schools with the top students have the strongest 
academic cultures. Among schools serving similar 
students, those with the stronger academic cultures 
have stronger student outcomes. 

In addition to strong personal relationships between 
students and teachers, we know that there are other 
important factors that positively influence student 
achievement, including the extent to which the school 
climate inculcates school-wide academic press for 
the future and importance of high school for the 
future. See sidebar for examples of questions. In 
schools where there is strong school-wide academic 
press for the future, freshmen have 1.9 fewer days of 



absence per semester, 0.47 fewer Fs, and their grade 
point averages are 0.15 points higher than in similar 
schools serving similar students but with weak school- 
wide academic press for the future. 

Not only do several aspects of the school climate 
affect how well students perform during their freshman 
year but these factors also continue to influence stu- 
dents’ experiences and their outcomes throughout high 
school and beyond. Moving from the freshman year to 
the senior year, we also have plenty of evidence showing 
how the school culture affects whether students enroll 
in four-year colleges after high school graduation. From 
High School to the Future: Potholes on the Road to College 
provided a detailed description of student efforts to 
enroll in four-year colleges. Of course multiple factors 
are involved here, with student qualifications being 
highest on the list. But even among highly qualified 



3. Importance of High School for the Future examines 
students’ evaluations of the importance of high school 
and what they are learning for their futures. High 
levels indicate that students recognize that high school 
is important for their future success in the workplace 
and for their life after graduation. 

How much do you agree with the following: 

• My classes give me useful preparation for what I 
plan to do in life. 

• High school teaches me valuable skills. 

• Working hard in high school matters for success 
in the work force. 

• What we learn in class is necessary for success in 
the future. 

Possible Answers: Strongly Disagree, Disagree, 
Agree, Strongly Agree 



4. Expectations for Postsecondary Education 

measures the extent to which teachers report that 
they feel responsible for encouraging and prepar- 
ing students for postsecondary education. Do they 
expect most students to go to college and focus 
on getting students ready for college? High levels 
indicate that the teachers in the school have high 
expectations of students’ postsecondary education. 

Please mark the extent to which you disagree or 
agree with the following: 

• Teachers expect most students in this school to 
go to college. 

• Teachers at this school help students plan for 
college outside of class time. 

• The curriculum at this school is focused on 
helping students get ready for college. 

• Most of the students in this school are planning 
to go to college. 

• Teachers in this school feel it is part of their job 
to prepare students to succeed in college. 

Possible Answers: Strongly Disagree, Disagree, 
Agree, Strongly Agree 



Chapter 2 



15 



students in CPS, many do not make it to college; and 
many who do end up in lower quality institutions where 
they have little chance to obtain a degree. 

One of the strongest factors influencing college going 
is the degree to which the high school has a strong col- 
lege going culture. We measure this concept on a scale 
called expectations for postsecondary education 
hy asking teachers a series of questions about whether 
they expect students to go to college, whether they help 
students and feel it is part of their job, and whether 
the high school curriculum is focused on college going. 
See sidebar on page 15 for all survey questions. Holding 
other factors constant, students with similar qualifica- 
tions were 12 percentage points more likely to plan to 
attend a four year college if they attended a high school 
with a strong college going culture than if they attended 
a high school with a weak college going culture. The 
difference was even greater for Latino students (15 
percentage points) who are overall less likely to plan 
to attend a four year college. The positive influence of 
having a strong college going culture in high school 
extends to the actual choices that students make about 
where to attend college. As noted earlier, many well 
prepared students end up attending weak institutions, 
where they flounder. Again, holding qualifications 
constant, students who attend a high school with a 
strong college going climate are 17 percentage points 
more likely to attend a college that better matches their 
qualifications than students in high schools with lower 
college going culture. 

Across multiple studies and data sources we con- 
tinue to find a great deal of variation in the amount 
of “value added” that schools contribute to students 
with similar backgrounds. As a general rule, schools 
that add the most positive values to their students’ 
outcome measures are schools with a strong academic 
climate and culture. This report only addresses selected 
aspects of a positive high school academic culture — stu- 
dent-teacher trust, school-wide academic press for the 
future, importance of high school for the future, and 
expectations for postsecondary education — and these 
only in relationship to certain outcomes, with some at 
the beginning of the high school experience (freshman 
year success) and others at the end of high school (four 
year college going) . 



The Impact of Academic Culture and 
Good Grades on ACT Scores 

Academic culture in high school also has a positive 
impact on ACT scores. As shown in the ACT prepa- 
ration study, schools where students report a strong 
culture of school-wide academic press for the future 
(see sidebar on pages 14-15) and teachers report a strong 
college going culture, ACT test scores are significantly 
higher, after controlling for student and school fac- 
tors, including students’ prior eleventh grade PLAN 
scores. On both the ACT Reading and Science tests, 
the difference in scores between schools strong on the 
two academic climate measures, compared to schools 
that are weak, is approximately 0.6 points of the ACT 
scale. At first thought, this may not seem like much of 
a difference, but note that the analysis controlled for 
the eleventh grade PLAN scores. On average, students’ 
scores improve less than one point between these two 
tests. This indicates that students who started eleventh 
grade with similar PLAN scores but attended high 
schools that were either strong or weak on these two 
important measures of academic climate ended up the 
junior year with ACT scores that differed by more than 
one-half of a point on a narrow scale where each point 
matters a great deal. There are similar differences on 
the Math and English subtests of the ACT. 

On its own, a strong academic climate spurs students 
on. Relationships between teachers and students are 
stronger, there is greater academic press on students, 
students learn that doing well in high school does 
matter for the future, and their teachers encourage and 
support their interest in going to college and help them 
get there. But these probably do not directly result in 
students in schools with stronger academic climate 
doing better on many academic outcomes, including 
ACT scores. It is more likely that these conditions 
facilitate greater student learning — students are more 
willing and perhaps able to invest in themselves with 
greater effort. Similarly, it is also likely that the stron- 
ger academic climate in schools is related to stronger 
instruction, where teachers make greater efforts and 
have higher aspirations for their students. And there is 
probably a greater payoff for both students and teachers 
in these schools with stronger academic climates as they 
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see the benefits and “return on investments” in the form 
of better achievement, including higher test scores. 

The ACT preparation study also showed the power- 
ful effect that getting good grades has on ACT scores. 
The researchers examined test score growth from the 
fall of eleventh grade (using PLAN scores) to spring 
eleventh grade ACT scores in all four subject areas. 
They found that, regardless of initial PLAN scores, 
students made significantly greater improvements on 
ACT as their grades went up. First we will see what 
happens with students who scored 17 on their eleventh 
grade Reading PLAN score. This is only marginally 
higher than the CPS average. Students who got As 
in eleventh grade English averaged 18.5 on the ACT; 
students who got Bs averaged 18.0; C grades resulted 
in ACT scores of 17.2; and students with Ds or Fs in 
English actually received ACT scores lower than their 
earlier PEAN scores. These analyses controlled for 
other factors, so we are confident that the good grades 
reflect student learning in the subject matter that then 
also becomes apparent on the ACT performance. 
Figure 7, which is adapted from the ACT report, 

FIGURE 7 

Students who started with an eleventh grade PLAN score of 
17 did better on ACT Math when they earned higher grades 
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shows the powerful effect of getting good grades on 
Math ACT test scores. Although this is not shown, this 
relationship holds for the other three subjects on the 
ACT. The finding is robust no matter what the starting 
PEAN score is. Students who are relatively weak on 
their PEAN scores but do well in class make big gains 
on the ACT. So do students who are strong on PEAN. 
No matter where they start on PEAN, students who 
do poorly in class actually lose ground on the ACT. 
Their scores slip backwards. These findings tell us that 
students need to be learning, growing, and developing 
in their classes and when they do they get a pay-off in 
terms of grades and test scores, both of which matter 
in their futures. 

Returning to the “pathway to 20,” we bring together 
these two important factors — strong academic culture 
and good grades to show their combined effects. We 
start with the ninth grade EXPEORE and return to 
discussing the ACT composite scores rather than the 
separate subject scores. We are looking at students 
with three different EXPEORE starting scores — first 
a 14, then a 17, and then a 19. Fourteen is one point 
below the CPS average, but still very typical for CPS. 
Seventeen is the score that we referred to earlier as the 
“tipping point,” where students start to have a very high 
likelihood of making it to an ACT score of 20. And 
finally, 19 is a strong EXPEORE score and students 
who score this high should make strong progress, end- 
ing with ACT scores that will make them eligible for 
competitive colleges where they should succeed. 

We will start with our students who are at the “tip- 
ping point” on EXPEORE with a composite score of 
17. There ends up being a big difference in ACT scores 
among these students depending on the strength of the 
academic culture in their high school and how good 
their grades were. Students who attended high schools 
with a strong academic culture (defined for this analysis 
as being above average on school-wide academic press 
for the future and in the top quarter of schools on ex- 
pectations for postsecondary education) and earned a 
grade point average of 3.0 or better during freshman, 
sophomore, and junior year advanced 3.7 points on the 
EPAS scale to receive an average ACT score of 20.7, 
past our goal of 20. See Figure 8. On the other hand, 
less fortunate students who also started out with a 17 
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FIGURE 8 

Students who earn high grades and attend strong schools see 
the biggest EXPLORE to ACT improvements from freshman to 
junior year 
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on EXPLORE but who went to weaker high schools 
and earned less than a B average only scored an average 
of 18.4 on the ACT. The students with the stronger 
academic climate and better grades gained more than 
twice as much on the EPAS system as their peers in 
the weaker schools and with the lower grades. The 
first group gained 3.7 points and the second group 
only 1.4. 

Now we turn to students who started high school 
with a 14 EXPLORE, which is low both in terms of 



national averages and CPS averages. Among these stu- 
dents, those who attended high schools with a strong 
academic culture and earned a grade point average of B 
or better in the freshmen, sophomore, and junior years 
advanced a full 3.0 points on the EPAS scale to reach 
17 on the ACT. This doesn’t make it to 20, but it is 
much closer than the scores of students who attended 
schools with weak academic culture and earned less 
than a 3.0 grade point average. These students gained 
only 1.2 points and ended up with an average ACT 
score of 15.2. The students who experienced stronger 
schools and earned better grades again gained more 
than twice as many points on EPAS than their peers 
in weaker schools and with lower grades. 

Einally, we look at students who scored 19 on 
EXPLORE in the fall of ninth grade. Eor CPS, this 
is a very strong EXPLORE score. Students who 
experienced a strong academic culture and earned 
good grades advanced nearly five full points on the 
ACT to an average scores of 23.9, whereas the less 
fortunate students improved by only 1 .9 points to an 
average ACT score of 20.9. 

So we see that regardless of the EXPLORE starting 
score, students who experience the stronger academic 
culture and earn better grades do much better on the 
ACTs than their less fortunate peers. In fact, under 
the better conditions, students improve by a factor of 
2.5 times as much as students do under worse condi- 
tions. More students get higher ACT scores when they 
attend strong high schools as we have defined them 
and earn good grades. 
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Interpretive Summary 

G iven the overall paucity of high ACT scores, CPS has set a goal of 
increasing the number of students who score 20 or better on their 
ACT. Even though we noted that this goal is somewhat lower than the ACT 
subject area benchmark scores, the more students who reach this goal, the 
more likely it is that they will be accepted into a four year college and be 
better prepared for college coursework. Though other studies have suggested 
that preparation alone doesn’t guarantee college access for many students, it 
surely is an important and necessary first step. 

The analyses and the data here show that ACT scores are influenced by 
multiple factors — prior achievement as measured by test scores, the academic 
culture of high school, and the quality of student work in high school as mea- 
sured by grades. These three factors are certainly interrelated. For example, it 
is easier to create to strong academic culture in a high school where students 
are better prepared; better prepared students have an easier time rising to the 
challenge to engage in academically demanding work and thereby earning top 
grades; and academic cultures are easier to build and sustain when teachers 
and students see results in terms of better grades and test scores. 

Yet we have strong evidence showing that schools and their academic 
cultures and the work that students do inside those schools pay off whether 
students look strong or weak in terms of their preparation. Even students 
with low test scores (on EXPLORE or PLAN) show substantial learning in 
stronger schools when they do better in their class work. Prior test scores are 
highly predictive of ACT scores, but students only improve on EPAS when 
they do the academic work necessary to get good grades. Students who start 
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out strong on tests and do poorly in their classes actu- 
ally fall back on ACT performance. Yes, prior scores 
matter, but more important is how those abilities are 
engaged in school and coursework. 

Aiming to increase the number of students who 
score 20 really focuses attention on a key outcome. 
But it also suffers the same problem as any criterion 
based on a single cut score — whether it is “the national 
norm,” or “state standards.” In order to move this 
indicator, there is a tendency to target extra resources 
on the students who have the best chance of moving 
over the cut score. These are often called the “bubble 
kids.” In this case, that would be students who scored 
17 on ninth grade EXPLORE. Of course we want 
more of these students to make it to 20, but not at the 
expense of other students whose EPAS gains are then 
imperiled. CPS needs more students making greater 
gains from EXPLORE to PLAN to ACT, whether 
they are within range or are reaching 20 or above or 
below. How then do we create incentives to move more 
students to higher ACT scores without more gaming of 
the system or without encouraging counterproductive 
test preparation activities? 

One way to do this would be to reward growth on 
the EPAS and to set reasonable growth goals, student- 
by-student. Yes, we want more students who score 
17 on EXPLORE to reach 20 on ACT, but at the 
same time we want more students who start with an 
EXPLORE score of 14 to reach 17, instead of 15. It 
is also just as important to have more of the strongest 
students — those with EXPLORE scores of 18, 19, and 
20 — move even higher. Perhaps, instead of using the 
“expected growth” metric as a standard for improve- 
ments on EPAS, we should be expecting more students 
to perform like their peers who experience a strong 
academic culture and get high grades. As reported 
here, we have the empirical evidence to show that this 
is possible in CPS. Such a goal setting system would 
challenge each student at his or her level and constantly 
ratchet up expectations, not by expecting everyone to 
reach a uniform and specific criterion but by having 
individualized goals that expect growth equal to the 
amount of growth that similar students achieve under 
the best conditions. 



More students can achieve like those under the best 
conditions — schools with strong academic cultures 
and the kind of instruction that students respond to 
with greater effort and engagement that will result 
in more learning, hence better grades and better test 
scores. The analyses described here suggest that strong 
academic cultures and high quality course instruc- 
tion and student performance go hand-in-hand. This 
further suggests that efforts to improve either culture 
or instruction should consider the close and probably 
reciprocal relationship between them. 

It seems necessary to reflect back on Eigure 5 (see 
page 11) that showed the relationship between eighth 
grade ISAT Math scores and the likelihood of reach- 
ing 20 or better on the ACT three years later. That 
graph shows the cut scores that the Illinois State Board 
of Education “adjusted” in 2006 to determine what 
defines “meeting state standards.” This cut score is so 
low that students who score just above it have only a 
slight chance of making the ACT 20 when they are 
juniors. The strongest academic cultures and the best 
grades aren’t going to be sufficient for these students 
who simply do not have the academic preparation to 
engage in the rigorous instruction they need to be 
prepared for college. Having such low academic stan- 
dards in eighth grade serves no one well, least of all the 
students who eke through and then are surprised to 
find themselves unprepared to do well in high school, 
let alone college. 

Simply raising standards for students in CPS or 
state- wide is not a solution, either. Even now we see very 
strong students — well into the Exceeds category — who 
do not reach even 20 on the ACT. This is a less an 
indictment of the standards than an indication that 
there are strong students who are being ill served by 
their high schools. Why else are even 15 or 20 percent 
of the highest scoring students not reaching a low bar 
on the ACT? We should have high expectations for our 
schools as well as for students. 

Our expectations for strong performance by all 
students need to start early in the elementary grades, 
if not in preschool. We should continue to use our 
technical capacities to develop information systems 
that identify students who need an academic nudge 
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throughout the grade levels. This is not for predict- 
ing who will fail hut for pointing out students who 
can succeed, just like many of their peers, if they 
receive the help that good schools and good teachers 
can provide. 

Finally, we need to acknowledge that there are no 
simple answers about how best to create a strong aca- 
demic climate under difficult conditions where failure 
has been the historic norm. This is the quandary facing 



CPS and its partners in the “Turnaround” schools^? 
and in other efforts to make drastic improvements in 
chronically struggling schools. We do not have answers, 
but we do know that there are many schools serving 
similar student populations where the outcomes are 
strong. It is our task as researchers and practitioners to 
learn more about the actual mechanisms involved in 
creating stronger climates where teachers and students 
work in partnership to achieve success. 
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Chapter 1 Endnotes 

1 Roderick, Nagaoka, and Allensworth (2006) 

2 Roderick, Nagaoka, Coca, and Moeller (2008) 

3 Allensworth, Correa, and Ponisciak (2008) 

4 Chicago Public Schools (2008) 

5 It is somewhat unfair to compare CPS or Illinois scores to the 
national average given that all students in Illinois (and Chicago) 
take the ACT, whereas the national sample is more selective. 

That is, for the most part students in other parts of the country 
take the ACT voluntarily and academically stronger students are 
more likely to select in. 

6 ACT, Inc. (2006) 

7 ACT, Inc. (2005); see www.act.org/research/policymakers/pdf/ 
benchmarks.pdf for a brief description of the benchmarks. 

For more detail, see the ACT Technical Manual. 

8 The Education Trust (2005) 

9 A higher percentage of CPS graduates score 20 or better on the 
ACT. Among the graduates of 2006, 30 percent scored 20 or 
better. This discrepancy occurs because more lower scoring stu- 
dents drop out after taking the ACT in the spring of junior year. 

A corollary disturbing fact is that even some higher scoring juniors 
do not complete high school. We estimate that between 9 1 and 93 
percent of students scoring 20 or better will graduate, leaving 7 to 
9 percent of these capable students without high school diplomas. 

1 0 Lane Tech, Whitney Young, Lincoln Park, Northside College Prep, 
and Payton College Prep. 

11 Chicago Public Schools (2007) 

12 ACT, Inc. (2008); for more information about EPAS, see 
www.act.org/epas/. 

13 CPS also administers EXPLORE to all eighth graders. Previously 
eleventh grade students also took the PLAN each fall. This practice 
is being discontinued in the fall of 2008. 

1 4 See page 22 of the EXPLORE Technical Manual for a discussion of 
what ACT means by being “approximately on the same scale.” 

1 5 See page 33 of Erom High School to the Euture: ACT Preparation — 
Too Much, Too Late. 



16 See Table 4.16 on page 34 in the EXPLORE Technical Manual. 

17 ACT, Inc. (2007c); see Table 4.9 on page 31. 

1 8 The very low scoring students have such high expected gains 
because their initial scores are below chance and may result 
from not completing the test. These students are likely to do 
much better on the next test. 

19 The expected gain for an EXPLORE score of 15 is 1.0. Our 
average expected gain of 1.3 is higher because both the lower 
scoring and the higher scoring students have expected gains 
that exceed 1.0. 

20 We observe the same pattern when we break down the EPAS scores 
more finely. We tracked back the average PLAN and EXPLORE 
scores for all students with ACT scores between 1 1 and 30. There 
is little variability in EXPLORE to PLAN gains across the range 
of ACT scores. There is great variability from PLAN to ACT, with 
students who end up with low ACT scores making little or no 
gains and the higher scoring students making greater gains. 

21 Our sample is reduced again, now down to 40,099. These are 
our 2005, 2006, and 2007 eleventh graders with ACT scores 
who also had eighth grade ISAT scores three years earlier. 

22 See Koretz (2008) for a good description of common standard 
setting processes. 

23 Banchero, Little, and Rado (2007) 

Chapter 2 Endnotes 

24 We get a lot of questions and comments about the subjectivity of 
grades and grade inflation. There is no doubt that these factors 
influence grades to some extent. On average across the entire 
system, however, grades and test scores are highly correlated. 

At the school level, there is a strong correlation between average 
grade point average and average ACT scores. In lower performing 
schools, relatively few students earn high grades. 

Chapter 3 Endnotes 

25 The “Turnaround” schools effort is attempting to improve student 
performance by replacing a school’s leader and staff 
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